BACKGROUND
During the course of my UFO investigations
and general reading I became interested in the quoted percentage of UFO reports
that were evaluated as 'unknown'. Figures in the 5 - 10% range (ie 90-95%
'IFO's) seemed to be common and generally accepted, although I had come across
much lower IFO figures.
It seemed to me that I needed a clearer
understanding of:
To this end, at the suggestion of and with the help of Philip Mantle, I instituted a survey of 18 UFO groups worldwide, plus the alt.alien.visitors and alt.paranet.ufo newsgroups.
THE SURVEY
Besides the obvious 'What's your percentage?' question, I also posed
a number of other questions relating to factors that I felt might influence
reported percentages, as shown below (with supporting detail where necessary):
What is your organisation's percentage of IFOs (Identified Flying Objects)?
How many reports is this percentage derived from, and in what year did you
start investigating reports?
[Obviously, more weight can be given to a percentage figure
based on 10,000 reports gathered over 40 years than 100 reports from a group
started up this year.]
How thoroughly is each report investigated?
[Someone sitting at a desk (dare I say wearing a USAF uniform,
in the 1960s?!) thumbing through a pile of reports and saying ''Venus, aircraft,
Venus, hmm, maybe we'll look into that one . .'' is taking a different route to
a conclusion on a report than someone who gets out there and investigates. I
would suggest that the former has a greater incentive to return an 'identified'
conclusion than the latter.]
Is the field investigator's explanation of a case taken as final, or
could this conclusion be overruled by a more experienced investigator before the
case is archived?
[Clearly the 'evaluator' has a duty
to 'filter' investigation reports handed in by inexperienced, enthusiastic
investigators (like yours truly!), but there's a risk that 'genuine
unidentifieds' (whatever that means) may be 'sanitised'. I don't somehow feel
there's a risk of a mis-evaluation going the other way. See UFORUM's
answer to this one.]
For IFOs, could you give a rough breakdown of the percentages of types of
IFO (eg stars, planets, aircraft etc)?
[I was just
curious as to what type of IFO was reported most often.] Have these proportions
changed over the years? [I was wondering whether, for man-made IFOs in
particular, people might be becoming more 'educated' over the years, and would
be mis-reporting this type of IFO less often. See MUFON's answer to this
one.]
What happens to cases where there is 'insufficient data'? Are they:
[If you do the maths, the answer to this one could have quite a significant effect on the reported percentages. In the UFO Experience, Hynek [2] shows that Blue Book's IFO percentage comes down to 77% if 'insufficient data' cases are counted as 'UFOs'.]
THE RESULTS
I eventually received six replies, including one Internet
message. For what they're worth, the 'raw' IFO percentage figures I got back
were:
| CUFOS: | (no figure given) |
| MUFON: | 80-90% |
| SOS OVNI: | 98% |
| UFO Research Australia: | 96-98% |
| UFOROM: | 31.5% |
| VUFORS: | 90% |
(The UFOROM figure is not a mistake - see UFOROM's entry.)
Other
relevant comments, by organisation, are as follows.
CUFOS
(J.Allen Hynek Center for UFO Studies.)
Disappointingly for such an august body, they had no statistics for
their IFO percentage, or for the total number of sighting reports on their
files. Whilst making some interesting observations (eg 'The real change that
has been noticed over the years is the dramatic world-wide decline in good UFO
reports.'), none of my survey questions were answered properly. (Reply
unsigned.)
MUFON
(This one from Walt Andrus himself!) Started investigating reports in
1967, but no total number of investigations given. Extensive network of
Investigators/State Directors/Regional Directors. Definite IFO reports (where
'IFO' could be decided at State Director level) not archived. Stated that
the number and proportion of IFO reports is moving down as the public
becomes more 'educated'. 'Insufficient data' cases not counted.
SOS OVNI
(France, P.Petrakis. Sexy stamps on this one!)
Didn't answer the questionnaire as such, because he (?) admitted that SOS OVNI
didn't keep any statistical records. Had been investigating UFO reports since
1974, returning 98% as IFOs. 'Insufficient data' cases counted as IFOs (50%
of the IFO conclusions coming from this source - see Table 1 for comments on
this topic). Cited common (peculiarly French?) IFOs as 'Sky Trackers' used to
advertise nightclubs, and rocket launches from the Biscarosse launch site.
UFO Research Australia
(aka Australian International UFO
Flying Saucer Research Inc.) Claimed to have investigated 60 cases since
1952 (??). 'Insufficient data' cases counted as UFOs. Very little information
in this reply, none of it useful (or believable!).
UFOROM
(Email from Chris Rutkowski.) A very interesting
and useful reply, easily the best that I received . Said that IFOs to them were
positively identified objects, and as such constituted a small
percentage of reports. Their 1993 figure was 31.5% (based on 489 reports) [3],
this being much higher than in previous years (due to a significant
fireball event occurring over Canada in October 1993). All reports within
UFOROM's 'patch' investigated very thoroughly. Case reports reviewed by a
'compiler', who could make 'adjustments' to the field investigator's
conclusions. 'Insufficient Data' cases kept as a separate category, i.e. not
counted as IFOs.
VUFORS
(Victorian UFO Research Society, Australia.) Quite
a useful report from Paul Norman. VUFORS has been investigating UFO reports
since 1957. Stated that 'best' cases such as the Frederick Valentich case and
the Knowles case were investigated very thoroughly, whereas 'lights in the sky'
cases were usually handled by post + telephone, although this policy was only
adopted after experience showed that follow-up visits in such cases usually
yielded little useful extra information. Gave a breakdown of IFO types, which,
interestingly, was not even close to the breakdown reported by Allan Hendry [1]
following his study; see Table 2. 'Insufficient data' cases not counted at
all. Sent extra information about a recent spate of 'prank balloons'
(reported as UFOs) in the Dandenong hills (ah!, the Dandenongs . . .).
DISCUSSION
The number of replies received was far to few
to draw any firm conclusions regarding the topics mentioned at the start of this
report. However, the fact that their was considerable variation in investigation
and reporting methods adopted by the replying organisations highlights the fact
that statements like '95% of UFO reports turn out to be reports of identifiable
objects or phenomena' should not be accepted unreservedly. Two particularly
interesting points to come out of the survey were:
These aspects are discussed below.
The treatment of 'insufficient data' cases
The table
below illustrates the effect of categorising 'insufficient data' cases as IFOs,
UFOs, or 'uncounted'.
Table 1: UFO Percentages
|
ID cases counted as. . . Reporting sample | IFOs | UFOs | (not counted) |
| Hendry (2.8% ID's) | 8.6 | 11.4 | 8.9 |
| Hynek/Blue Book (18% ID's) | 5.5 | 23.5 | 6.7 |
| JTAP ('80-'82) [4] (28% IDs) | 14.6 | 42.4 | 20.3 |
| UFOROM ('89-'93) (41% IDs) | 13.1 | 54.1 | 22.2 |
| SOS OVNI (49% IDs) | 2.0 | 51 | 3.9 |
Here we see that, for example, for the Blue Book sample, counting ID cases
as 'identified' (the usual approach) gives a UFO proportion of 5.5%; counting
the ID cases as 'unidentified' (ie as UFOs, as suggested by Hynek) gives a UFO
proportion of 23.5%; not counting the ID cases at all (ie by saying that, in
this example, there were not 10137 cases but (10137-1822) = 8315 cases, 557 of
them unidentified) gives a UFO proportion of 6.7%.
Two things to note here:
The latter may just result from a mathematical property of the data (ie forget we're talking about UFOs at all) or it could be saying that a wide variation exists in the way 'insufficient data' (and, by implication, the other categories) are defined.
The breakdown of IFO types
The table below illustrates
the differing breakdown of IFO types (definite IFOs, IDs not counted) across
three samples. I was just idly curious about this, but I began to wonder if
there may be significant differences in the availability of some IFO
types to different groups in different parts of the world at different times.
For example, 'Ad planes' were Hendry's second-most frequent reported
IFO; 'ad planes' are virtually unknown in the UK, especially at night, so what
would Hendry have done if he went on to investigate the same number of UFO
reports in the UK? Would the temptation to assign reports to the other
(globally-available) IFO types have been stronger than it should have been, thus
resulting in a few 'real' UFOs being missed?
All hypothetical I know,
but I still think the table below makes interesting reading.
Table 2: IFO Breakdowns
| %age of
total IFOs IFO Type | Hendry | VUFORS | JTAP |
| Stars & planets | 35.2 | 75.0 | 25.4 |
| Advertising planes | 22.5 | ||
| Aircraft | 19.1 | 8.0 | 26.3 |
| Meteors and re-entries | 11.0 | | 15.2 |
| Ball lightning | 5.9 | ||
| Satellites | 2.3 | 4.2 | |
| Moon | 2.1 | 1.7 | |
| Prank balloons | 1.4 | ||
| Searchlights | 0.9 | ||
| Balloons | 0.9 | 2.0 | |
| Missile launches | 0.6 | ||
| Fixed ground lights | 0.5 | <1.0 | |
| Flares | <0.5 | 3.4 | |
| Hoaxes | - | 5.0 | 1.7 |
| Hallucination | 1.7 | ||
| Birds | <0.5 | ||
| Kites | <0.5 | 1.7 | |
| Clouds | <0.5 | <1.0 | |
| Test clouds | <0.5 | ||
| Airborne Residue | <0.5 | <1.0 | |
| Mirage | <0.5 | ||
| Moondog | <0.5 | ||
| Window reflection | <0.5 |
RECOMMENDATIONS
I would have liked more data, but what I
think the survey shows is that there is considerable variability in the
percentages of UFO report evaluation categories, and that this variation is due,
at least in part, to the different definitions used for each evaluation
category.
If all report evaluation agencies used the same definitions,
then one could be reasonably confident that any differences in evaluation
patterns thrown up by the different groups would be due to differences in the
data itself rather than the way it was evaluated.
What I think is
needed therefore are standard definitions of:
Positively Identified
The reported object has been identified with 100% certainty.
For example, the investigator contacts an airline (or air force)
which says: 'Yes, the serial number you gave us is one of ours, and it was in
the air at that location at that time.'
Probably/Possibly Identified
The reported object has all, or some, of the characteristics of its
probable/possible identification.
An example of 'probable' would
be a case where the investigator is as near as dammit certain that an aircraft
has been reported, but the independent verification which would result in a
positive identification cannot be obtained.
An example of 'possible' would be a case where the investigator
suspects that an aircraft has been reported, and that those characteristics
which do not seem to be ascribable to an aircraft are the result of
mis-reporting. (Witnesses could be confused by 'nocturnal light' sightings,
where lack of a fixed frame of reference (the ground) makes it difficult to tell
whether an object is moving along or across the observer's line of sight.)
Insufficient Data
Some or all basic details such as the observer's location, the time and
date, and the observed object's appearance and movement are missing from the
report.
Unidentified
Despite an intensive investigation, and despite there being sufficient
information to otherwise evaluate the report, no probable or possible
explanation in terms of known phenomena can be suggested.
Report
The investigator has spoken to a witness of a sighting and has recorded
details of the conversation.
Thus an investigator could be 'fed' details of a sighting from a
variety of sources, or the investigator could track down a witness, starting
from, say a newspaper report. A newspaper report in itself thus would not count
as a report.
WHERE DO WE GO FROM HERE?
Adoption of the above
definitions is not going to solve the UFO mystery, but if all investigators used
them then any subsequent analysis of evaluation profiles (across different
investigation agencies) would be meaningful.
Of course, when I say 'all investigators' then it would be wonderful if that could mean worldwide! In practice though, that is probably going to be impossible to achieve (although perhaps some inroads could be made through ICUR), but I think that adoption of a standard set of definitions within BUFORA at least, should be achievable.
We already have the recent requirement that no-one can become an Accredited Investigator for BUFORA unless he/she successfully completes the investigator's postal training course; this gives reason to believe that the quality of investigations will increase. Use of a standard set of definitions for evaluation purposes should increase the consistency of evaluations.
I suggest that use of a standard set of definitions should be mandatory; investigators should be directed not to return a UFO conclusion (for example) unless the 'UFO' conclusion definition is met. If BUFORA can arrive at a situation where high-quality, consistent investigations are being produced then the workload on the Director of Investigations would be reduced (although he/she should still have the right to overrule investigators' conclusions if necessary).
A standard set of definitions could be enshrined in an all-new BUFORA Investigators Manual.
. . . now there's a thought . . .
REFERENCES